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EXECUTIVE  SUMMARY 


This  report  addresses  some  of  the  issues  that  arise  when  simulation 
is  used  to  evaluate  capacity-enhancing  yair  traffic  control  (ATC) 
system  changes  that  may  also  affect  -^safety.  It  examines  the 
limitations  of  different  approaches'  to  the  measurement  and 
prediction  of  safety,  j 

A  safety  standard  is  proposed  that  is  not  based  on  meeting  a 
predetermined  or  absolute  criterion,  but  on  the  ability  to 
demonstrate  that  the  modified  system  is  as  safe  as  the  present  one 
—  assuming  the  present  system  is  considered  safe.j; 

If  the  intent  of  change  is  to  improve  system  safety  rather  than  to 
increase  capacity,  this  approach  would  require  showing 
significantly  safer  operations  vis-a-vis  the  present  standard  and, 
perhaps,  no  loss  of  capacity. 


This  approach  is  based  on  the  concept  of  ATC  simulation  as  a 
controlled  experiment.  It  provides  a  defense  against  having 
results  contaminated  by  a  lack  of  complete  realism  in  the 
simulation,  by  a  paucity  of  hard  data  on  the  occurrence  of  system 
errors,  and  by  the  difficulty  of  setting  a  standard  for  an 
acceptable  probability  of  serious  events.  ^ 

£. 


v 


INTRODUCTION 


BACKGROUND. 

SAFETY  CRITERIA  FOR  SYSTEM  CHANGE.  Questions  of  system  safety  and 
the  interpretation  of  data  are  complex.  The  writer  believes  the 
absolute  requirement  for  modifying  standard  procedures  is  the 
demonstration  of  undiminished  safety.  This  can  be  done  in  a  number 
of  ways: 

1.  Demonstrate,  through  the  collection  and  analysis  of  operational 
data,  that  present  standards  are  unnecessarily  restrictive. 

2.  Conduct  flight  tests  proving  the  feasibility  and  safety  of 
proposed  changes. 

3.  Conduct  operations  research,  math  modeling,  or  fast  time 
simulation  and  examine  the  impact  of  proposed  changes  on  a  variety 
of  operational  parameters  and  contingencies. 

4.  Conduct  real  time  air  traffic  control  (ATC)  simulation  studies 
of  the  present  and  the  changed  system,  introduce  errors  and 
failures,  and  compare  the  results. 

These  approaches  are  neither  independent  nor  mutually  exclusive. 
Reliable  field  data  are  essential  for  successful  modeling  and  for 
simulation.  Real  time  ATC  and  flight  simulation  and  flight  testing 
are  needed  to  generate  estimates  of  the  operational  parameters  used 
for  modeling  and  fast  time  simulation.  Modeling  provides  a 
framework  for  collecting  and  analyzing  field  data. 

The  desire  to  provide  absolute  certainty  in  the  outcome  of  an 
extremely  rare  event  may  reduce  system  capacity  below  acceptable 
limits,  or  worse,  produce  new  and  higher  risks  in  other  areas  now 
considered  safe. 

Ultimately,  it  falls  to  experienced  system  users  (controllers, 
pilots,  operations  personnel,  etc.)  to  weigh  the  sometimes  con¬ 
flicting  evidence  from  these  sources  and  make  the  decision,  based 
on  their  understanding  of  (1)  day  in,  day  out  operations,  (2)  the 
knowledge  and  skills  of  the  controllers,  and  (3)  kinds  of  contin- 
.  gencies  the  system  must  respond  to. 

This  report  will  limit  itself  to  only  one  of  the  many  different 
activities  that  come  under  the  general  heading  of  ATC  Simulation. 
It  addresses  simulations  with  the  following  characteristics:  (1) 
controllers  use  plan  view  displays  and  voice  communications  to 
control  simulated  aircraft;  (2)  simulator  pilots  respond  to,  and 
reply  with  standard  phraseology;  (3)  simulated  aircraft  exhibit 
realistic  flight  dynamics;  (4)  flight  plans,  flight  profiles, 
traffic  flow,  and  procedures  are  appropriate  to  the  airspace  under 
study. 
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For  many  applications,  ATC  simulation  provides  a  useful  model  of 
the  "real  world."  Sophisticated  simulation  equipment  permits 
traffic  levels,  aircraft  dynamics,  and  sector  geography  to  be 
matched  to  today's  or  tomorrow's  systems.  Still,  there  are  always 
differences.  ATC  simulator  pilots  are  not  usually  professional 
pilots  and  they  may  control  five  or  more  aircraft  at  a  time;  there 
is  no  analog  of  cockpit  workload;  realistic  weather  is  difficult 
to  include;  simulation  equipment  and  procedures  often  require 
compromises;  participating  controllers  may  have  to  work  with 
unfamiliar  equipment,  proce-  dures  and  airspace;  and  if  blunders 
or  anomalies  are  introduced  as  part  of  safety  testing,  they  soon 
become  anticipated. 

METHODOLOGICAL  ISSUES.  Whether  or  not  these  differences  are 
important  depends  on  the  purposes  of  the  simulation.  Even  with  the 
limitations  men-  tioned,  simulation  offers  a  good  projection  of 
potential  changes  in  system  capacity  and  controller  workload.  This 
is  especially  true  when  current  operations  are  tested  with  changes 
in  routes  or  procedures,  and  the  same  controllers  can  participate 
in  "before  and  after"  tests. 

The  evaluation  of  system  safety  places  additional  demands  on 
simulation  testing.  Three  distinct  issues  are  raised;  simulation 
conditions,  safety  criteria,  and  safety  standards. 


DISCUSSION 

SIMULATIONS  CONDITIONS. 

In  a  fairly  realistic  simulation  environment  the  rare  events  that 
adversely  affect  safety  might  not  ever  occur,  even  in  a  long  study. 
They  certainly  would  not  happen  often  enough  for  systematic 
investigation.  Years  of  testing  would  be  needed  if  results  had  to 
be  stated  in  terms  of  events  "per  million  opera-  tions"  or  "per 
million  hours." 

Several  things  can  be  done  to  insure  the  occurrence  of  the  rare 
events  that  are  necessary  for  thorough  system  testing.  These 
include: 

1.  Deliberate  pilot  noncompliance  or  miscompliance. 

2.  Dimulation  of  equipment  errors  and/or  failures. 

3.  The  use  of  unusually  high  traffic  rates  to  maximize  pressure 
on  the  controllers. 

While  the  introduction  of  rare  or  unusual  events  is  essential  to 
safety  testing,  their  impact  on  system  performance  is  complex. 
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Even  safe  systems  can  be  expected  to  have  problems  with  unusual 
events,  occasionally  serious  problems.  Decision  making  must 
evaluate  not  only  the  danger  in  an  outcome,  but  its  likelihood. 
This  is  especially  important  when  two  rare  events,  with  independent 
likelihoods,  are  made  to  occur  simultaneously. 

SAFETY  CRITERIA. 

Safety  criteria  are  system  measures  and  observations  indicative  of 
the  safety  of  the  system  being  tested.  Some  of  the  usual  measures 
are: 


1.  Counts  of  conflicts  or  separation  violations  that  occur. 

2.  Ratings  by  controllers  and  observers  of  system  safety  (using 
notes,  questionnaires  or  debriefing  after  each  run  or  series  of 
runs)  . 

3.  Various  measures  and  indices  of  aircraft  proximity,  such 

as  slant  range  miss  distance  and  the  Aircraft  Proximity  Index 
(API)  . 

For  consensus  to  be  reached  on  the  interpretation  of  simulation 
results,  there  must  be  agreement  on  the  choice  of  safety  criteria. 

If  the  critical  safety-threatening  incident  in  parallel  runway 
operations  is  the  30°  blunder,*  the  criteria  must  measure  the 
ability  of  the  controllers  using  the  system  to  provide  safe 
resolution.  This  measure  should  be  based  on  the  resulting 
separation  of  affected  aircraft. 

CONFLICTS .  The  definition  of  a  conflict  depends,  in  part,  on  the 
airspace  and  operations.  Generally,  a  conflict  occurs  when 
separation  between  two  aircraft  is  less  than  3  miles  laterally  and 
less  than  1000  feet  vertically  in  terminal  airspace,  or  less  than 
5  miles  and  less  than  1000  feet  (2000  feet  above  29,000  feet)  en 
route.  There  are  many  exceptions,  such  as  when  one  pilot  sees  the 
aircraft  ahead  and  accepts  visual  separation,  or  both  aircraft  are 
established  on  parallel  localizers. 

The  occurrence  of  conflicts  is  not  a  useful  criteria  in  evalu¬ 
ating  parallel  operations,  since  conflicts  are  an  unavoidable 
result  of  the  blunder. 

Two  other  criteria  have  been  used  in  recent  studies:  Slant  Range 
Miss  Distance  (SRMD)  and  the  Aircraft  Proximity  Index  API. 


*  A  blunder  is  defined  as  an  unexpected  turn  by  a  plane,  already 
established  on  the  localizer  and  cleared  to  land,  towards  an 
adjacent  parallel  approach. 
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Each  is  computed,  second-by-second,  for  each  aircraft  pair  in  a 
conflict,  with  a  single  value,  the  smallest  for  SRMD,  the  largest 
for  API,  assigned  to  the  conflict. 

API  follows  the  definition  of  a  conflict  in  considering  that  3 
nautical  miles  (nmi)  (18,228  feet)  of  horizontal  distance  and  the 
1000-foot  vertical  distance  each  provide  safe  separation.  This 
makes  1  foot  of  vertical  distance  equivalent  to  18  feet  of  lateral 
separation. 

A  comparison  of  the  three  measures  is  shown  in  figure  1.  The 
standard  definition  of  a  conflict  can  be  visualized  as  a  flat 
cylinder  surrounding  an  aircraft,  extending  1000  feet  above  and 
below,  and  3  miles  in  radius.  SRMD  treats  separation  as  a  sphere 
around  an  aircraft,  with  the  same  dimension  in  all  directions. 
API  can  be  thought  of  as  a  concentric  set  of  discus-shaped  layers 
surrounding  an  aircraft  (a/c) ,  circular  in  shape  and  tapering  in 
thickness  from  the  center  to  the  edge.  Figure  2  contains  a  set  of 
cross  sections  of  API  values  drawn  to  scale.  Figure  3  is  a  set  of 
3-dimensional  plots  for  APIs  of  1,  25,  50  and  75.  In  both 
illustrations,  one  aircraft  can  be  considered  the  target  at  the 
center,  while  the  location  of  the  intruder  determines  the  API. 
The  values  are  symmetrical  around  either  aircraft. 

SRMD.  When  a  conflict  occurs  —  usually  by  the  introduction  of  a 
blunder  —  the  slant  range  distance  of  the  a/c  pair  is  computed 
each  second.  The  least  distance  between  the  centers  of  gravity  of 
the  two  aircraft  defines  the  SRMD.  SRMD  can  be  thought  of  as  the 
radius  of  the  smallest  observed  sphere,  centered  on  one  of  the 
aircraft,  the  surface  of  which  passes  through  the  center  of  gravity 
of  the  other  aircraft. 

API .  A  detailed  discussion  of  API  is  contained  in  the  appendix. 
This  index  was  developed  by  the  author  and  has  been  reported  and 
used  in  several  Technical  Center  simulations  (reference  1,  2,  3, 
and  4)  . 

Computation  is  as  follows: 

Dv  =  vertical  distance  between  a/c  in  feet 
Dh  =  horizontal  distance  in  nmi  (6,076') 

API  =  ( 1 , 000-DV) 2  * ( 3 -DH) 2/(90,000) 

While  it  may  not  be  obvious  from  the  formula,  API  can  also  be 
expressed  as  the  product  of  squared  proportions  of  LOST  standard 
separation,  normalized  from  0  to  100: 


Api  =(l-  DV  f  *  (l-  °H  ¥  *  100 
'  10007  3 
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CONFLICT  BOUNDARY 


B.  Slant  Range  Miss 
Distance  Boundaries 


<3 .  API  BOUNDARIES 


APPROXIMATE  SCALE 


FIGURE  1.  COMPARISON  OF  THREE  SAFETY  CRITERIA. 
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FIGURE  3.  3-DIMENSIONAL  BOUNDARIES  FOR  FOUR  VALUES  OF  API. 


Figure  3.  3-D  views  of  the  boundaries  of  four  values  of  API. 

as  seen  in  cross  section.  The  vertical  scale  of  these  plots 
is  enlarged  for  clarity.  The  target  a/c  is  centered  on  the 
base.  The  raised  surface  shows  all  possible  locations  of 
another  a/c  such  that  the  API  will  equal  the  designated  value 
Labels  around  the  base  are  in  1  nmi increments . 


An  example  is  a  60  percent  loss  of  horizontal  and  vertical 
separation: 

API  =/l-  400  \ 2  */l-  1 . 2  \  2  *  100  OR  .62  *  .  62  *  100  =  12.96 
\  1000^  \  3  / 

API  differs  from  SRMD  is  several  respects: 

1.  The  maximum  API,  100,  occurs  when  the  centers  of  gravity  of 
two  aircraft  are  collocated.  API  is  considered  to  be  0  and  not 
calculated  ^or  horizontal  separations  more  than  3  nmi  OR  vertical 
separations  of  1000  feet  or  more.  The  highest  calculated  value 
becomes  the  measure  of  the  conflict.  Representative  API  values  are 
shown  in  table  A-4  in  the  appendix. 

2.  API  is  not  linear  with  respect  to  distance.  A  linear  decrease 
in  distance  between  the  aircraft,  either  vertically  or  laterally, 
increases  the  API  exponentially. 

3.  API  treats  the  horizontal  distance  between  the  aircraft  and 
the  vertical  separation  independently. 

An  SRMD  of  500  feet  can  generate  an  API  ranging  from  95  (0  feet 
vertical,  500  feet  horizontal  distance)  to  25  (500  vertical,  0 
horizontal  distance)  (see  figure  A-4  in  the  appendix.) 

As  an  example,  an  SRMD  of  500  feet  could  be  produced  by  two 
aircraft  that  miss  each  other  by  500  feet  on  intersecting  courses 
at  the  same  altitude.  If  the  aircraft  were  flying  at  170  knots, 
or  almost  287  feet  per  second,  they  would  pass  through  the 
intersection  less  than  2  seconds  apart.  The  API  would  be  95. 

Another  500-foot  SRMD  example  involves  path  intersection  with  only 
vertical  separation.  With  one  aircraft  flying  the  glide  slope  at 
170  knots  (15  feet  per  second  descent  rate),  the  other  flying  level 
500  feet  below,  it  would  take  more  than  30  seconds  for  them  to  be 
at  the  same  altitude.  The  API  would  be  25. 

STANDARDS . 

Standards  are  required  for  reaching  decisions  concerning  the  re*l 
world  that  are  based  on  the  criteria.  A  standard  should  be  a  set 
of  rules  or  procedures,  agreed  upon  before  the  simulation  starts, 
that  determine  how  the  data  collected  lead  to  proper  conclusions. 

One  possible  standard  might  be,  "A  system  is  not  safe  if  any 
simulated  event  produces  a  clearly  unsafe  outcome."  Another 
possibility  is,  "If  a  class  of  events  that  can  be  expected  to  occur 
no  more  than  once  every  5  years  in  the  real  world  leads  to  an 
unacceptable  outcome  in  the  simulation  no  more  than  one  time  in 
100,  the  system  can  be  considered  safe." 
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The  alternatives  demonstrate  some  of  the  difficulties  in  pro¬ 
viding  indisputable  evidence  of  system  safety.  Every  decision  on 
conditions,  criteria,  and  standards  is  capable  of  affecting 
results.  Also,  there  are  elements  of  subjectivity  and  judgment  in 
designing  a  simulation  as  there  are  in  creating  and  evaluating 
deterministic  or  probabilistic  models. 

An  article  by  Poritzky  and  Horowitz  (reference  5)  addressed  several 
issues  relevant  to  the  question  of  standards.  They  point  out  that 
while  "Elaborate  models  have  been  constructed. . .neither  the 
industry  nor  the  FAA  have  found  a  way  to  validate  [them]  because 
they  are  derived  from  abstract  representations  with  multiple 
assumptions  about  the  dependence  or  independence  of  the  elements, 
and  only  shaky  knowledge  of  coprobabilities. 

As  a  result,  the  application  of  failure  probability  estimating 
models  should  be  viewed  with  skepticism  for  any  problem  other  than 
comparing  alternatives  in  relative  terms." 

With  respect  to  safety  standards,  they  say,  "There  are  two  separate 
problems:  (1)  problems  associated  with  establishing  the  numerical 
value  for  a  safety  standard,  and  (2)  problems  associated  with 
demonstrating  compliance  with  the  standard  that  has  been 
established.  The  latter  problem  may... be  insurmount-  able." 

But,  under  the  heading  "Establishing  an  ’Objective'  Standard  of 
Safety,"  they  state,  "On  the  face  of  it,  the  establishment  of  a 
standard  should  not  be  difficult.  The  number  should  be  no  lower 
than  the  present  risk  of  an  aircraft  accident  as  evidenced  by 
available  statistics"  (emphasis  added) . 

While  the  safest  standards  are  preferred,  there  is  usually  some 
trade-off  between  safety  standards,  system  capacity,  and  system 
costs.  These  trade-offs  don't  lend  themselves  to  simple  analytical 
solutions . 

IMPROVING  CAPACITY  WHILE  MAINTAINING  SAFETY.  Capacity  changes  can 
be  evaluated  with  real-time  simulation,  fast-time  models,  or 
analytically.  Simulation  is  important  if  the  proposed  improvement 
may  affect  the  controllers'  job.  In  the  case  of  multiple  parallel 
runways,  each  runway  monitor  performs  the  same  basic  task, 
regardless  of  the  number  of  run  ways  or  monitors  involved.  Other 
things  being  equal  (which  is  unlikely) ,  airport  capacity  should 
increase  with  additional  independent  instrument  runways. 

Demonstrating  the  maintenance  of  safety,  however,  requires 
simulation  —  unless  there  is  a  proven  fast-time  simulation  or 
analysis  of  unquestioned  validity  that  is  applicable  to  the  changes 
being  proposed. 
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What  is  needed  is  an  objective  criterion  that  can  be  used  to 
evaluate  system  safety.  While  absolute  standards  for  system  safety 
are  exceedingly  difficult  to  establish,  a  pragmatic  standard  can 
serve.  The  one  proposed  here  is  based  on  the  premise:  SYSTEM 
CHANGES  SHOULD  NEVER  REDUCE  SAFETY.  That  is,  today's  operation  — 
if  considered  safe  —  can  be  a  standard  against  which  to  test 
proposed  changes.  In  experimental  terms,  the  present  operation, 
whether  it  is  the  number  of  parallel  runways  or  the  distance 
between  them,  is  run  as  the  "control,"  while  the  system  to  be 
tested,  either  multiple  runways  or  closer  spacing,  is  the 
"experimental  condition." 

All  pertinent  operational  conditions  must  be  included  in  the 
simulation.  In  the  study  of  parallel  approaches  this  includes  an 
appropriate  mix  and  number  of  aircraft,  realistic  flight  technical 
error,  appropriate  radar  characteristics  and  air/  ground 
communications,  and  the  occurrence  of  incidents  or  equipment 
failures  the  system  should  be  able  to  manage  safely. 

The  same  controllers  run  the  same  traffic  at  the  same  consoles  and 
with  comparable  system  errors  introduced  under  both  conditions. 
The  decision  standard  asks  whether  the  experimental  condition  is 
as  safe  as  the  control  condition. 

If  enough  runs  are  made  with  enough  controllers  and  traffic,  if 
the  tests  are  carefully  controlled  to  avoid  extraneous  factors, 
and  if  the  measures  of  safety  are  relevant  (valid)  ,  it  will  be 
possible  to  reach  a  conclusion  on  the  safety  of  proposed  system. 
It  may  be: 

1.  The  experimental  condition  is  not  as  safe  as  the  control,  or 

2.  The  experimental  condition  is  as  safe  as  the  control,  or 

3.  The  data  are  insufficient  to  answer  the  question  with  the 
necessary  precision. 

The  ability  to  generalize  from  the  simulation  to  the  real  world  is 
possible  because  the  proposed  and  present  systems  are  evaluated 
under  identical  conditions,  criteria,  and  standards.  Only  if  there 
is  some  artifact  in  the  simulation  that  differ-  entiallv  affects 
the  two  systems  is  a  "wrong  answer"  likely. 


*  The  consoles  might  be  different  if  changes  in  displays  were 
part  of  the  new  system  under  consideration. 
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IMPROVING  SAFETY  WHILE  MAINTAINING  CAPACITY.  The  approach  outlined 
above  can  be  applied  to  the  improvement  of  safety  as  well  as  its 
maintenance.  Using  a  similar  experi-  mental  design,  only  the 
hypothesis  being  tested  changes.  The  questions  to  be  answered 
might  be  phrased  as  follows: 

1.  The  experimental  condition  is  safer  than  the  control. 

2.  The  experimental  condition  is  not  safer  than  the  control. 

3.  The  data  are  insufficient  to  answer  the  question  with  the 
necessary  precision. 

The  safety  standard  proposed  here  can  be  applied  whether  SRMD,  API, 
or  both  are  used  as  criteria.  The  standard  requires  con-  ducting 
simulation  runs  using  the  control  (the  present  day  operations)  and 
the  experimental  (proposed  system)  conditions  while  introducing 
large  numbers  of  system  errors  or  blunders. 

The  controllers  always  attempt  to  provide  the  safest  separation 
they  can  in  each  case,  and  their  effectiveness  is  measured  by  the 
SRMDs  or  the  APIs  generated  by  the  ensuing  maneuvers.  Since  the 
same  controllers  work  under  all  conditions,  any  statistically 
significant  differences  in  SRMD  or  API  must  be  attributed  inherent 
safety  differences  between  the  conditions. 

It  has  been  pointed  out,  that  one  does  not  normally  PROVE  the  "null 
hypothesis,"  i.e.,  that  there  are  no  differences  between 
experimental  conditions.  To  be  confident  that  the  experimental 
condition  is  not  less  safe  than  the  standard,  there  must  be  some 
assurance  that  the  simulation  was  adequate  to  detect  a  difference 
if,  in  fact,  it  existed.  Success  here  depends  on  the  care  used  in 
planning  and  executing  the  study.  But  it  can  be  confirmed  if 
significant  differences  are  found  among  variables  of  known 
importance.  For  parallel  runway  studies,  these  include  factors 
such  as  the  blunder  angle,  initial  aircraft  separations,  and  the 
presence  or  absence  of  communications. 

SUMMARY 

The  standard  described  above  frees  experimenters  and  decision 
makers  from  the  time-consuming,  expensive,  and  ultimately  futile 
pursuit  of  complete  realism  in  simulation.  It  is  still  necessary 
to  know  which  conditions  can  effect  the  outcome  and  assure  they 
are  included  at  appropriate  levels. 

The  fact  that  some  rare  events  result  in  dangerous  outcomes  will 
not  be  as  important  as  the  differences  in  outcomes  between 
conditions.  This  is  especially  true  as  long  as  the  system  errors 
or  blunders  are  truly  rare  events. 

On  the  other  hand,  if  actual  system  errors  were  frequent  and  their 
outcomes  generally  dangerous,  a  "safe  standard"  would  be  difficult 
to  find. 
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APPENDIX  A 


AN  AIRCRAFT  PROXIMITY  INDEX  (API) 


AN  AIRCRAFT  PROXIMITY  INDEX  (API) 


BACKGROUND . 

Air  Traffic  Control  (ATC)  Simulation  is  an  essential  research  tool  for  the 
improvement  of  the  National  Airspace  System  (NAS) .  Simulation  can  never 
offer  all  of  the  complexity  and  subtlety  of  the  real  world,  with  live  ra¬ 
dar,  actual  aircraft,  full  communications  systems  and  the  rest  of  the  ATC 
environment,  but  it  can  provide  an  intensive  exercise  of  key  portions  of 
the  system  --  with  controllers  in  the  loop. 

Proper  use  of  simulation  starts  with  carefully  defining  the  questions  to  be 
answered  and  then  developing  a  simulation  environment  which  includes  the 
features  that  could  influence  the  process  under  study.  The  selection  of  a 
simulation  environment,  the  development  of  scenarios,  the  choice  of  data  to 
be  recorded,  and  the  method  of  analysis  are  part  science,  part  art. 

An  important  benefit  of  simulation  is  that  it  permits  the  exploration  of 
systems,  equipment  failures,  and  human  errors  that  would  be  too  dangerous 
to  study  with  aircraft,  or  that  occur  so  rarely  in  the  system  that  they 
cannot  be  fully  understood  and  evaluated.  A  current  example  of  this  use 
has  to  do  with  the  introduction  of  blunders^  in  parallel  runway  instrument 
approaches . 

The  introduction  of  large  numbers  of  system  errors  is  a  useful  way  to  study 
safety,  but  the  analysis  of  the  outcomes  of  these  incidents  is  not  always 
simple  or  clear  cut. 

SAFETY  EVALUATION. 

1.  Conflicts. 

The  occurrence  of  a  conflict  in  normal  ATC  operations  is  considered  prima 
facie  evidence  of  a  human  or  system  error.  Identifying  (and  counting)  con¬ 
flicts  under  a  variety  of  conditions  is  one  way  to  expose  a  system  problem. 

A  conflict  is  defined  as  the  absence  of  safe  separation  between  two 
aircraft  flying  under  Instrument  Flight  Rules  (IFR) .  At  its  sim¬ 
plest,  safe  separation  requires:  (a)  the  aircraft  must  be  laterally 
separated  by  3  or  5  nmi,  depending  on  distance  from  the  radar,  (b) 
vertical  separation  by  1,000  or  2,000  feet,  depending  on  altitude 
or  flight  level,  or  (c)  that  both  aircraft  are  established  on  ILS 
localizers . 

There  are  refinements  of  the  above  rules  that  take  into  considera¬ 
tion  the  fact  that  one  aircraft  may  be  crossing  behind  another,  or 
that  an  aircraft  has  begun  to  climb  or  descend  from  a  previous  al¬ 
titude  clearance.  There  are  also  special  "wakes  and  vortices"  re¬ 
strictions  for  aircraft  in  trail  behind  heavy  aircraft. 


1A  blunder  is  defined  as  an  unexpected  turn  towards  an  adjacent 
approach  by  an  aircraft  already  established  on  the  Instrument 
Landing  System  (ILS) . 
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Since  actual  conflicts  are  rare,  every  event  leading  up  to  them  and  all  the 
information  available  on  the  onset  and  resolution  is  carefully  analyzed. 

The  emphasis  is  on  the  intensive  investigation  of  the  particular  event. 

In  scientific  investigation,  the  intensive  study  of  a  single  individual  or 
a  particular  event  is  called  the  idiographic  approach.  This  is  often  con¬ 
trasted  with  the  nomothetic  approach:  the  study  of  a  phenomenon  or  class  of 
events  by  looking  at  large  numbers  of  examples  and  attempting  to  draw  gen¬ 
eral  conclusions  through  the  application  of  statistics. 

The  idiographic  approach  is  mandatory  for  accident  or  incident  investiga¬ 
tion  where  the  goal  is  to  get  as  much  information  as  possible  about  a 
unique  event  in  order  to  prevent  future  occurrences . 

In  a  simulation  experiment,  where  the  goal  is  to  make  a  comparison  between 
two  or  more  systems  (2  vs  3  or  4  runways,  4300  vs  3000 -foot  runway  spacing, 
etc.)  and  to  generalize  beyond  the  simulation  environment,  the  nomothetic 
approach  is  most  appropriate.  This  means  generating  a  large  numbers  of 
events  and  statistically  analyzing  the  outcomes  with  respect  to  the  system 
differences . 

There  is  much  to  be  gained  by  studying  the  individual  conflicts  in  a  simu¬ 
lation  as  an  aid  to  understanding  the  kinds  of  problems  that  occur  and  to 
generate  hypotheses  about  how  a  system  might  be  improved  for  subsequent 
testing.  But  the  evaluation  of  the  systems  under  test  requires  the  use  of 
all  of  the  valid  data,  analyzed  in  as  objective  a  manner  as  possible. 

Valid  data  in  this  context  means  that  it  was  collected  under  the  plan  and 
rules  of  the  simulation  and  was  not  an  artifact,  such  as  a  malfunction  of 
the  simulation  computer  or  distraction  by  visitors. 

2.  Slant  Range. 

If  it  is  important  to  go  beyond  the  counting  of  conflicts,  measurement  of 
the  distance  between  the  conflicting  aircraft  pair  is  required.  The  most 
obvious  measure  is  slant  range  separation:  the  length  of  an  imaginary  line 
stretched  between  the  centers  of  each  aircraft.  Over  the  course  of  the 
incident  that  distance  will  vary,  but  the  shortest  distance  observed  is  one 
indication  of  the  seriousness  or  danger  of  the  conflict. 

The  problem  with  slant  range  is  that  it  ignores  the  basic  definition  of  a 
conflict  and  is  insensitive  to  the  different  standards  that  are  set  for 
horizontal  and  vertical  separation.  A  slant  range  distance  of  1100  feet 
might  refer  to  1000  feet  of  vertical  separation,  which  is  normally  per¬ 
fectly  safe,  to  less  than  0.2  nmi  of  horizontal  miss  distance,  which  would 
be  considered  by  most  people  to  be  a  very  serious  conflict.  Slant  range, 
per  se,  is  too  ambiguous  a  metric  to  have  any  real  analytical  value. 

3.  API. 

The  need  exists  for  a  single  value  that  reflects  the  relative  seriousness 
or  danger.  The  emphasis  here  is  on  "relative,"  since  with  the  nomothetic 
or  statistical  approach,  an  absolute  judgment  of  dangerous  or  safe  it  use¬ 
ful,  but  not  sensitive  enough.  The  requirement  is  to  look  at  the  patterns 
of  the  data  for  the  different  experimental  conditions  and  determine  whether 
one  pattern  indicates  more,  less,  or  the  same  degree  of  safety  as  another. 
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Such  an  index  should  have  to  have  certain  properties. 

a.  It  should  consider  horizontal  and  vertical  distances  separately, 
since  the  ATC  system  gives  18  times  the  importance  to  vertical  separation 
(1,000  ft  vs  3  nmi) . 

b.  It  should  increase  in  value  as  danger  increases,  and  go  to  zero 
when  there  is  no  risk,  since  the  danger  in  the  safe  system  is  essentially 
indeterminate . 

c.  It  should  have  a  maximum  value  for  the  worst  case  (collision),  so 
that  users  of  the  index  can  grasp  its  significance  without  tables  or  addi¬ 
tional  calculations. 

d.  It  should  make  the  horizontal  and  vertical  risk  or  danger  inde¬ 
pendent  factors,  so  that  if  either  is  zero,  i.e.,  safe,  their  product  will 
be  zero. 

e.  It  should  be  a  nonlinear  function,  giving  additional  weight  to  se¬ 
rious  violations,  since  they  are  of  more  concern  than  a  number  of  minor  in 
fractions . 

The  API  is  designed  to  meet  these  criteria.  It  assigns  a  weight  or  value 
each  conflict,  depending  on  vertical  and  lateral  separation.  API  facil¬ 
itates  the  identification  of  the  more  serious  (potentially  dangerous)  con- 
flictions  in  a  data  base  where  many  conflictions  are  present.  One  hundred 
has  been  chosen,  somewhat  arbitrarily,  for  the  maximum  value  of  the  API. 

APPROACH . 

During  a  simulation  API  can  be  computed  whenever  a  conflict  exists.  For 
convenience,  this  is  taken  to  be  when  two  aircraft  have  less  than  1,000 
feet  of  vertical  separation  AND  less  than  3.0  nmi  of  lateral  separation. 

It  is  computed  once  per  second  during  the  conflict.  The  API  of  the  con¬ 
flict  is  the  largest  value  obtained. 

API  considers  vertical  and  horizontal  distances  separately,  then  combines 
the  two  in  a  manner  than  gives  them  equal  weight;  equal  in  the  sense  that  a 
loss  of  half  the  required  3.0  nmi  horizontal  separation  has  the  same  effect 
as  the  loss  of  half  the  required  1000  feet  of  vertical  separation. 

COMPUTATION. 

The  API  ranges  from  100  for  a  mid-air  collision  to  0  for  the  virtual 
absence  of  a  technical  confliction.  A  linear  decrease  in  distance  between 
the  aircraft,  either  vertically  or  laterally,  increases  the  API  by  the 
power  of  2. 

Computation  is  as  follows: 

Dy  -  vertical  distance  between  a/c  (in  feet) 

D^  -  horizontal  distance  (nmi  (6,076')) 


To  simplify  its  use,  API  is  rounded  off  to  the  nearest  integer,  i.e., 

API  -INT( (1 , 000-Dv)2*(3-Dh)2/(90 , 000)+. 5) 

The  rounding  process  zeros  API's  less  than  0.5.  This  includes  distances 
closer  than  2  nmi  AND  800  feet.  The  contour  plot  in  figure  A-l,  demon¬ 
strates  the  cutoff  for  API  -  1. 

See  tables  A-l  and  A- 2  for  typical  values  of  API  at  a  variety  of  distances. 

Figure  A-2  is  a  three-dimensional  plot  showing  the  relationship  between  API 
and  vertical  and  horizontal  separation  graphically.  Figure  A- 3  shows  the 
same  information  in  a  slightly  different  way.  Anything  outside  the  contour 
at  the  base  is  "0" .  In  figure  A-4  a  contour  plot  of  API  for  horizontal  and 
vertical  distances  from  0  to  500  feet  is  shown,  with  300-foot  and  500-foot 
slant  range  distances  superimposed. 

DISCUSSION. 

The  index  is  not  intended  as  a  measure  of  acceptable  risk,  but  it  meets  the 
need  to  look  at  aircraft  safety  in  a  more  comprehensive  way  than  simply 
counting  conflictions  or  counting  the  number  of  aircraft  that  came  closer 
than  200  feet,  or  some  other  arbitrary  value. 

It  should  be  used  to  compare  conflicts  in  similar  environments,  i.e.,  an 
API  of  70  in  en  route  airspace  with  speeds  of  600  knots  is  not  necessarily 
the  same  concern  as  a  70  in  highly  structured  terminal  airspace  with  speeds 
under  250  knots. 

Since  the  API  is  computed  every  second,  it  may  be  useful  to  examine  its 
dynamics  over  time  as  a  means  of  understanding  tie  control  process. 
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TABLE  1.  TYPICAL  VALUES 


VERTICAL 

DISTANCE  HORIZONTAL  DISTANCE  IN  NAUTICAL  MILES  (1  NM  -  6076' ) (DH) 


(Dv>  3 

(in  feet) 

2.5 

2.0 

1.5 

1.0 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

.05 

.01 

-0 

1000 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

900 

0 

0 

0 

0 

0 

0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

800 

0 

0 

0 

1 

2 

2 

2 

2 

3 

3 

3 

3 

3 

4 

4 

4 

4 

700 

0 

0 

1 

2 

4 

4 

5 

5 

6 

6 

7 

7 

8 

8 

9 

9 

9 

600 

0 

0 

2 

4 

7 

8 

9 

9 

10 

11 

12 

13 

14 

15 

15 

16 

16 

500 

0 

1 

3 

6 

11 

12 

13 

15 

16 

17 

19 

20 

22 

23 

24 

25 

25 

400 

0 

1 

4 

9 

16 

18 

19 

21 

23 

25 

27 

29 

31 

34 

35 

36 

36 

300 

0 

1 

5 

12 

22 

24 

26 

29 

31 

34 

37 

40 

43 

46 

47 

49 

49 

200 

0 

2 

7 

16 

28 

31 

34 

38 

41 

44 

48 

52 

56 

60 

62 

64 

64 

100 

0 

2 

9 

20 

36 

40 

44 

48 

52 

56 

61 

66 

71 

76 

78 

80 

81 

-0- 

0 

3 

11 

25 

44 

49 

54 

59 

64 

69 

75 

81 

87 

93 

97 

99 

100 

TABLE  2. 


°H 

DV 

API 

°H 

3.0 

1000 

0 

1.0 

3.0 

0 

0 

1.0 

0 

1000 

0 

1.0 

2.0 

667 

1 

1.0 

2.0 

500 

3 

1.0 

2.0 

333 

5 

1.0 

2.0 

250 

6 

.5 

2.0 

100 

9 

.5 

2.0 

0 

11 

.5 

1.5 

667 

3 

.5 

1.5 

500 

6 

.5 

1.5 

333 

11 

.1 

1.5 

250 

14 

.1 

1.5 

100 

20 

.  1 

1.5 

0 

25 

.1 

.1 

ADDITIONAL  VALUES 


°V 

API 

°H 

°V 

API 

667 

5 

.05 

667 

11 

500 

11 

.05 

500 

24 

333 

20 

.05 

333 

43 

250 

25 

.05 

250 

54 

100 

36 

.05 

100 

78 

0 

44 

.05 

0 

97 

667 

8 

.01 

667 

11 

500 

17 

.01 

500 

25 

250 

39 

.01 

333 

44 

100 

56 

.01 

250 

56 

0 

69 

.01 

100 

80 

667 

10 

.01 

0 

99 

500 

23 

0 

667 

11 

250 

53 

0 

500 

25 

100 

76 

0 

333 

44 

0 

93 

0 

250 

56 

0 

100 

81 

0 

0 

100 
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FIGURE  1.  CONTOUR  PLOT 

This  is  a  contour  plot  of  API.  showing  the  values  of  API  for  the  horizon¬ 
tal  separations  of  0  to  3  nmi,  and  vertical  separation  of  0  to  1,000  feet. 
Values  less  than  API  -  .5  round  to  zero.  This  includes  a/c  separated  by 
as  little  1.6  nmi  horizontally  AND  850  feet  vertically. 
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FIGURE  2.  3 -DIMENSIONAL  CONTOUR  PLOT 


Three-dimensional  contour  plot  of  API,  for  horizontal  separations  of  0  to 
3  nmi,  and  vertical  separations  of  0  to  1,000  feet. 
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FIGURE  3.  3-DIMENSIONAL  CONTOUR  PLOT 

Left  vertical  plane  shows  API  vs  horizontal  distance  with  vertical  dis¬ 
tance-0.  Right  vertical  plane  shows  API  vs  vertical  separation  with  hori¬ 
zontal  distance  -  0. 

Plot  may  be  interpreted  by  considering  one  a/c  at  the  center  of  the  base 
plane,  while  the  height  of  the  figure  shows  the  API  for  another  a/c  any¬ 
where  else  on  the  base  plane. 

The  contour  on  the  base  plane  shows  the  boundary  between  API  -0  and  API=1 . 
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API  VALUES  FOR  SLANT  RANGES  OF  300  AND  500  FEET 


0  100  200  300  400  500 


Lateral  Distance  in  Feet 

FIGURE  4.  CONTOUR  PLOT  OF  API  FOR  HORIZQNTAL  AND  VERTICAL 
DISTANCES  OF  0  TO  500  FEET,  SHOWING  SLANT  RANGE 
CONTOURS  OF  300  AND  500  FEET 

This  plot  shows  the  API  values  (the  small  numbers,  inside  the  square  run¬ 
ning  from  25  at  the  top  to  100  at  the  bottom)  for  equal  API  contours  (the 
slightly  sloping  horizontal  lines)  for  horizontal  and  vertical  distances 
of  0  to  500  feet.  API  values  range  from  25  (500-foot  vertical,  0  horizon¬ 
tal  separation)  to  100  (0/0) . 

The  500 -foot  slant  range  contour  has  API  values  ranging  from  25  to  95, 
depending  on  amount  of  vertical  component.  The  300-foot  slant  range  con¬ 
tour  runs  from  API  -  49  to  97.  Using  API  as  a  criterion,  500-foot  slant 
range  can  be  more  dangerous  than  300 -foot. 
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